Checkout counter

ABSTRACT

A classification device ( 2 ) for identification of articles ( 3 ) in an automated checkout counter is presented. The device comprises a memory unit ( 5 ) capable of storing digital reference signatures, each of which digital reference signatures corresponds to an article identity, a processor ( 6 ) connected to the memory unit ( 5 ), and at least one sensor ( 4, 7, 14, 15, 16, 17, 18, 24 ) configured to determine a measured signature of an article ( 3 ) wherein said processor ( 6 ) is configured to compare said measured signature with the digital reference signatures, and to calculate a matching probability of a predetermined number of article identities.

TECHNICAL FIELD

The present invention relates to a classification device for identifyingarticles in an automated checkout counter as well as to an automatedcheckout counter comprising such classification device. The inventionfurther relates to a method for an automated checkout counter.

BACKGROUND

In today's stores many different types of articles, e.g. vegetables,fruit, and different packages may be purchased. Some stores uses aclassification device of an automated checkout counter for identifyingthe articles. For example, U.S. Pat. No. 4,676,343A describes how to usea conveyor scales together with a laser scanner for reading labels at acheckout counter. The labels are particularly arranged to be read by thelaser scanner and comprises information about the weight of the articlewhich should be matched with the actual weight of the article as read bythe scales. A problem with the device according to U.S. Pat. No.4,676,343 is that the laser scanner and the scales must be used foridentifying the article, which is resource consuming and creates severalinterruptions in the process if one of the laser scanner or scalesshould fail. Another problem is that the particular label must bepresent on the article, which requires that the customer must attach alabel on the article if there is no label. Problems may therefore arisedue to wrong handling by the customer. The need of a label furtherprovides a drawback in that the customer may not easily purchase bulkarticles but will be required to weigh and identify the articles inorder to assure the correctness of the required label.

Other known devices are described in CA2054851, U.S. Pat. No. 5,662,190,US20060138220, and US20040262391.

Classification devices of checkout counters are thus well known, butnone of the previously known devices are automated for handlingdifferent types of articles such as fruit and packages and at the sametime being arranged to provide an optimum degree of security withrespect to identification, however still requiring a minimal use ofsensor resources.

SUMMARY

With respect to the prior art there is a need for an improvedclassification device of a checkout counter for automatic identificationof articles where the number of incorrect identifications is approachingzero, but where the sensor resources are used optimally for reducing theprocessor power such that a high processor speed is retained.

The present invention seeks to solve the above mentioned problems bymeans of a classification device for identification of articles.

According to a first aspect of the invention, a classification devicefor identification of articles in an automated checkout counter isprovided. The classification device comprises a memory unit capable ofstoring digital reference signatures, each of which digital referencesignatures corresponds to an article identity, a processor connected tothe memory unit, and at least one sensor configured to determine ameasured signature of an article wherein said processor is configured tocompare said measured signature with the digital reference signatures,and to calculate a matching probability of a predetermined number ofarticle identities.

According to a second aspect, an automated checkout counter is providedcomprising a classification device according to the first aspect.

According to a third aspect of the invention, a method for classifyingarticles in an automated checkout counter is provided. The methodcomprises the steps of providing a classification device comprising amemory unit capable of storing digital reference signatures, each ofwhich digital reference signatures corresponds to an article identity, aprocessor connected to the memory unit, and at least one sensorconfigured to determine a measured signature of an article, wherein themethod comprises the steps of comparing said measured signature with thedigital reference signatures, and calculating a matching probability ofa predetermined number of article identities.

According to a yet further aspect of the invention, an automatedcheckout counter comprising a classification device for identificationof articles is provided. The classification device comprises a weightsensor for weighing the article, a memory unit comprising information ofone or several articles, a processor connected to the memory unit and tothe weight sensor, as well as an infrared spectroscopy sensor, fromhereon denoted as a NIR sensor and detecting wavelengths fromapproximately 780 nm to 2500 nm, connected to the processor. The memoryunit comprises one or several first signatures created by the first NIRsensor or another NIR sensor, each of which first signatures isconnected to a corresponding article identity. The first signatures maybe created directly at the checkout counter by using the first NIRsensor, a second NIR sensor, or by storing signatures created by a NIRsensor not connected to the checkout counter in said memory.

When a NIR sensor is used on a certain kind of articles, e.g. a specifictype of apple, a first signature will be received which may be coupledto the article and which may be denoted as a specific article identityin the memory unit, like e.g. the name of the article. Each type ofarticle creates a unique first signature which may be coupled to theidentity of the article. The first NIR sensor is arranged to create asecond signature connected to the article when an article is placedbefore, on or after the weight sensor. The processor is subsequentlyarranged to compare the second signature to the first signature in orderto identify the article as an existing article identity in the memoryunit. The checkout counter is arranged to weigh the article by means ofthe weight sensor before, during or after the creation of the secondsignature. The weight of the article is subsequently used by theprocessor together with the article identity for determining the priceof the article.

An advantage of the present invention is that the checkout counter mayautomatically identify all kinds of articles without the need for acustomer to identify the article prior to the checkout counter, e.g. byattaching a bar code. The NIR sensor is particularly valuable foridentification of fruit and vegetables, as well as certain types of bulkarticles, since such articles have previously required that the customerhas identified the article and subsequently labeled it due to the factthat sensors using cameras and image processing have not been capable ofdetermining the identity of the article.

The weight sensor preferably comprises a conveyor scale whichautomatically conveys and measures the article. The customer will hereposition the article on the conveyor belt which either weights andsubsequently conveys, or conveys, holds, and weights, for laterconveying of the article. As previously been mentioned the first NIRsensor may be arranged at the checkout counter before, during, or afterweighing. The fastest way is however to allow the first NIR sensor toidentify the article when the conveyor belt holds for measuring thearticle.

According to one embodiment of the invention, as a complement to the NIRsensor and the weight unit, the checkout counter may be equipped withone or several sensors which, if they are used according to theinvention, provides the advantage of increased security when identifyingthe article but with a minimal use of resource and consequently time andenergy. It should be mentioned that an article should be identified inless than one second for considering an automated checkout counter to beuser friendly by the customers.

The present invention seeks to solve the problem of optimal use ofresources with several sensors by using the sensors according to certainpredetermined combinations which provides high security in identifyingthe article as well as high process speed. The combinations also providethe advantage that when a given combination is fulfilled, i.e. thesensor or sensors in combination provides a positive result ofidentification, the other sensors may be disconnected or directedtowards identification of another article, which provides anoptimization of the process resources.

The embodiment with several sensors is thus designed on a number ofpredetermined combinations comprising partial set of existing sensors,where it will be sufficient that one of the predetermined combinationsprovides a positive result. The sensors may be switch on, i.e. beactivated, in sequences in order to find beneficial combinations orpartial set of sensors or all sensors may be active until one of thecombinations provide a positive result. A positive result should in thiscontext be interpreted as if all sensors of the combination havedetected and identified a predetermined property of an article, whichproperties in combination provide an articles identity. The identity maybe determined by means of checking a database comprising properties of anumber of articles. Example of properties may include weight, size,color, shape, contour, marking by a barcode and/or text and/or figureand/or pattern.

According to the embodiment the classification device always comprises aweight sensor and a NIR sensor according to above, as well as one orseveral of: a contour sensor and/or a barcode sensor and/or a symbolreading sensor which uses optical character recognition and (machine)text interpretation and/or a color texture sensor and/or a colorhistogram sensor and/or a VIS sensor. The symbol reading sensor is fromhereon called OCR which is a general known abbreviation of the Englishterm “Optical Character Recognition”. The VIS sensor is a spectrometercomprising a light source and a VIS camera, from hereon called a VISsensor, the VIS sensor is detecting wavelengths from approximately 200nm to 1100 nm. The spectrum thus overlaps the wavelengths of visuallight which extends from 400 nm to 660 nm. Experiments have shown that,at the device according to the invention, the classification devicecomprising a color texture sensor and/or a color histogram sensor and/ora VIS sensor does not operate satisfactory when the VIS sensor isoperating in the complete frequency interval 200 nm-1100 nm since thereis a conflict between the color sensors and the VIS sensor in theinterval of visual light, i.e. between 400 nm and 660 nm.

The VIS sensor according to the invention is therefore active in theintervals between 200 nm and 400 nm and between 660 nm and 1100 nm whenit is combined with the color texture sensor and/or the color histogramsensor. If the color texture sensor and the color histogram sensor aredisconnected the VIS sensor may however operate in the completefrequency interval between 200 nm and 1100 nm since there is noconflict. The processor is programmed to control the sensors to achieveoptimal efficiency of the classification device.

The sensors cooperate in a way that if the sensors are activated in thefollowing combinations the remaining sensors, except the weight sensorand the NIR sensor, are allowed to be deactivated or not be activated atall, depending on the article being identified:

-   -   weight sensor and contour sensor and OCR, or    -   contour sensor and OCR, or    -   weight sensor and OCR, or    -   weight sensor and color histogram sensor and contour sensor, or    -   weight sensor and contour sensor and OCR, or    -   weight sensor and color histogram sensor and contour sensor and        color texture sensor and OCR and barcode sensor, or    -   weight sensor and contour sensor and color texture sensor and        OCR or    -   weight sensor and barcode sensor, or    -   only OCR, or    -   VIS sensor in combination with any of the above combinations, or    -   only VIS sensor.

One advantage of the invention is that the combinations provide anoptimal high security with a minimal use of resources, which will beexplained below.

The symbol reading sensor is connected to a computer/image processingunit which uses an algorithm using information from images from theexisting camera or cameras of the device. For articles, whichsubstantially can be unambiguously identified by means of symbolreading, it will be sufficient if the symbol reading sensor, OCR,identifies a symbol and/or a text which then unambiguously identifiesthe article. Examples of articles which may be identified by only usinga symbol reading sensor, OCR, are pre-packaged packages where thecustomer is not required to perform any procedure, such as refilling orany other procedure. Example of articles where it is not enough with thesymbol reading sensor, are some bulk articles where the quantity of thearticle, i.e. weight, is not known. Further properties of the articlemay be necessary and may require symbol reading and/or weight and/orcolor histogram and/or color texture and/or contour. It shall bementioned that “contour” is defined as a two dimensional projection of athree dimensional object.

Certain articles are thus more difficult to identify than others anddepending on the article one or several of the included sensors of theclassification device are required.

Preferably, the weight sensor comprises one conveyor scale comprisingone conveyor part and one weight unit connected thereto whichautomatically conveys the article, weighs it and transmit theinformation of the weight to the database. In this way contributionsfrom workers and customers will be eliminated which removes the need ofmanual transport of the article over the weight unit. One or severalsensors may be connected to the checkout counter for controlling theconveyor scale.

The contour sensor comprises a camera for providing still or movingimages and may preferably be a linear camera which reads a horizontallyprojected surface or a linear camera in combination with an objectsensor which consists of a vertical light curtain for reading thevertical projection. The contour sensor is connected to an imageprocessing unit where the contour, i.e. a two dimensional projection ofa three dimensional object, is checked against the properties in thedatabase.

The barcode sensor comprises a camera for providing still or movingimages. The barcode sensor is connected to an image processing unitwhere the barcode is checked against the properties in the database.

The symbol reading sensor comprises a camera for providing still ormoving images. The symbol reading sensor is connected to an imageprocessing unit where the symbol is checked against the properties inthe database.

The color texture sensor comprises a camera for providing still andmoving image. The color texture sensor is connected to an imageprocessing unit where the color texture is checked against theproperties in the database. The image processing unit comprises analgorithm which calculates where a certain color is present in theimage. One common algorithm is “Weibull color texture algorithm”, butother algorithms may also be considered.

The color histogram sensor comprises a camera for providing still andmoving pictures. The color ratio in the image is usually illustrated bymeans of a representation, a so-called histogram. A histogram isgenerated by examination of all pixels of the image, and the number ofpixels having a specific color value are summarized.

The above mentioned image processing units may consist of one or severalunits and may comprise one or several computers with software capable ofperforming the above mentioned analyzes. The classification device maycomprise one or several cameras which are included in the abovementioned sensors. One example of a preferred embodiment is that thecontour sensor comprises a first camera positioned in a way that thecontour is read when the article passes the camera. According to theinvention a linear camera is suitable since the reading then occursduring the conveying of the article between two conveyor belts or over atranslucent surface. It is also suitable that the classification devicecomprises a second camera and possibly several cameras to be able to seethe article from different angles for achieving the highest possiblereliability when detecting barcode, text and images. The other camera,and if applicable a further camera/cameras, is arranged to record animage or images which will be used by the image processing unit foranalyzes of color histogram, color texture, OCR and barcode reading. Onefurther alternative is that the classification device comprises only thefirst camera and the second camera where the second camera is opticallyconnected to one or several lenses which observe the article fromdifferent angles and where the image processing unit analyses the imagesfrom corresponding angles. The previously mentioned linear camera, beingpositioned between the conveyor belts, is however the only camera whichmay capture if the barcode is positioned downwards on the article.

The NIR sensor operates in such way that infrared light illuminates thearticle and the reflecting infrared light from the article is beinganalyzed with reference to phase displacement caused by surfaceratio/surface properties and chemical bonds at the article which createsa reflection spectrum. NIR sensors are known per se by prior art.

As mentioned above NIR is a shortening of the English term “NearInfraRed Spectroscopy” and comprises a light source for near infraredlight and a NIR camera which may register near infrared light. Nearinfrared light typically has a wavelength of 580-2500 nm, or preferably780-1750 nm. The wavelength has shown to be suitable for analyzing bulkmaterial, fruit and vegetables. In this context “NIR” may include thelight source and the NIR camera, i.e. the complete NIR arrangement foranalyzes. However, “NIR sensor” may only include the sensing equipment,e.g. the light guiding probe and the spectrometer.

By analyzing a known article with a NIR sensor a unique reflectionspectrum is received which may be connected to the article. Thereflection spectrum may either be used directly as a signature connectedto the article or the reflection spectrum is processed to create thesignature. An article in a store may look different at differentoccasions, e.g. an article grow old (eventually fruit will rotten) andthe article may be packed in one or several plastic bags or the articlemay be solitary or in a group, or being arranged in differentorientations; natural variations of the article occurs also, etc. Theenvironment for a checkout counter may also be different in differentstores, e.g. different amount of light, color, etc. All these parametersprovide that a NIR spectrum of a certain article in a certainenvironment at a certain occasion does not necessarily match withanother NIR spectrum of the article in another environment at anotheroccasion. To be able to use a NIR sensor at a checkout counter accordingto the invention the first signature has to match the second signatureat a certain degree such that the processor is able to identify thearticle by a comparison. It is thus an advantage if the first signatureis created in the same environment as the second signature. Since thesecond signature is created at the checkout counter during use, it is anadvantage if the first signature is created during the same conditions.According to the invention, the classification system has thus aself-learning functionality in which the first signature is created byprogramming the memory unit with an article identity whereafter thearticle is transported trough the checkout counter during circumstancessimilar to use, i.e. circumstances for the checkout counter which refersto customer use. To consider the mentioned variations the article istransported several times through the checkout counter and in differentvariations, e.g. with one or several bags and/or solitary or in a group,etc. Each time the article is transported through the checkout counterand a NIR sensor is analyzing the article a first signature is created,which means that each article identity may be connected to a largeamount of first signatures such that the processor will be able toidentify the article when comparing it to the second signature and oneor more of the first signatures. During learning the first NIR sensormay be arranged to perform the analyses, or a second NIR sensor will beconnected. The learning does not need to be performed at the exactlocation where the checkout counter will be used but may rather beperformed at another location.

At the creation of the first and second signature the surroundings willbe considered by means of a background spectrum, i.e. an empty checkoutcounter, or an empty conveyor belt . When analyzing an article thebackground spectrum is known and the processor may consider in differentways.

The linear camera of the contour sensor is preferably used incombination with the NIR camera to provide information of where thearticle is positioned on the belt. The NIR camera is movable along theslit between two conveyor belts but needs time to move into position forreading.

The VIS sensor is a spectrometer comprising a light device suitable forthe mentioned wavelengths and a VIS camera capable of registering lightof the wavelengths between 200 nm and 1100 nm. Similar to the NIR sensorthe VIS sensor uses the change in wavelength when light is partlyabsorbed by or reflected by an article. The VIS sensor is particularlysuitable for analyzing different shades of brown, which makes itsuitable for analyzing bread which is normally hard to classify by meansof any of the other sensors. The different shades of brown aredetectable by the VIS sensor.

In this context “VIS sensor” may include the light source as well as theVIS camera, i.e. the complete VIS device for analyzing. However, the VISsensor may also be a separate device, not connected to the light source,but including a light guiding probe and a spectrometer.

By analyzing a known article by means of a VIS sensor a uniquereflection spectrum, VIS spectrum, is received, which may be coupled tothe article. The reflection spectrum may either be used directly as asignature for the article, or the reflection spectrum may be processedfor creating the signature. An article in a store may look different atdifferent occasions, e.g. an article grow old (eventually fruit willrotten) and the article may be packed in one or several plastic bags orthe article may be solitary or in a group, or being arranged indifferent orientations; natural variations of the article occurs also,etc. The environment for a checkout counter may also be different indifferent stores, e.g. different amount of light, color, etc. All theseparameters provide that a VIS spectrum of a certain article in a certainenvironment at a certain occasion does not necessarily match withanother VIS spectrum of the article in another environment at anotheroccasion. To be able to use a VIS sensor according to the invention at acheckout counter a third signature, representing a background signature,and a fourth signature comprising background of article, has to matchsuch that the processor is able to identify the article by a comparison.It is thus an advantage if the third signature is created in the sameenvironment as the fourth signature. Since the fourth signature iscreated at the checkout counter during use, it is an advantage if thethird signature is created under the same conditions. According to theinvention, the classification system has thus a self-learningfunctionality in which the third signature is created by programming thememory unit with an article identity whereafter the article istransported trough the checkout counter during circumstances similar touse, i.e. circumstances for the checkout counter which refers tocustomer use. To consider the mentioned variations the article istransported several times through the checkout counter and in differentvariations, e.g. with one or several bags and/or solitary or in a group,etc. Each time the article is transported through the checkout counterand the VIS sensor is analyzing the article a third signature iscreated, which means that each article identity may be connected to alarge amount of third signatures such that the processor will be able toidentify the article when comparing it to the fourth signature and oneor more of the third signatures. During learning the first VIS sensormay be arranged to perform the analyses, or a second VIS sensor will beconnected. The learning does not need to be performed at the exactlocation where the checkout counter will be used but may rather beperformed at another location.

The VIS sensor may comprise a fiber cable acting as a probe whichdistributes light from the article to the VIS camera.

The NIR sensor may comprise a fiber cable acting as a probe whichdistributes light from the article to the NIR camera.

Each of the VIS sensor and the NIR sensor may be connected to a fibercable acting as probes which are arranged to converge into a commonfiber cable which distributes light from the article to the VIS cameraand the NIR camera.

The classification device may comprise a handheld barcode reader whichis connected to the database. The handheld barcode reader may be usedwhen articles are too big for the being conveyed on the conveyingdevice.

The classification device may advantageously comprise a self-learningfunction which admits that the system becomes self-learning.“Self-learning” means that all sensors of the classification devicebecome active for identification of an article when it passes thesensors for the first time. The sensors identify theproperties/characteristics of the article and store the properties inthe database. When the self-learning function is used the article isalready registered in an article register with a predetermined identity,e.g. EAN code, and optionally price. The article register is either apart of the database or a separate database connected to the database ofthe article identity.

The classification device may be complemented by a barcode readerconnected to the database and may preferably be used at theself-learning function. The first time the article is being transportedthrough the classification device the fixed scanner reads the barcodewhich guarantees the identification of the article, which leads to thatthe properties being detected by the sensors are being stored in thedatabase as the correct article identity.

The sensors may preferably be placed completely or partly in a tunnelshaped construction which shields a part of the conveyor belt andtherefore improves the security by preventing unauthorized people fromthe possibility to affect the classification process.

DESCRIPTION OF THE DRAWINGS

Hereinafter, the invention will be described with reference to a numberof drawings, wherein:

FIG. 1 schematically shows a top view of a checkout counter according toa first embodiment of the invention;

FIG. 2 schematically shows a side view of the checkout counter accordingto FIG. 1;

FIG. 3 schematically shows a top view of a checkout counter according toa second embodiment of the invention;

FIG. 4 schematically shows a side view of the checkout counter accordingto FIG. 3.

FIG. 5 schematically shows a top view of a checkout counter according toa third embodiment of the invention;

FIG. 6 schematically shows a side view of the checkout counter accordingto FIG. 5.

FIG. 7 schematically shows a top view of a checkout counter according toa fourth embodiment of the invention;

FIG. 8 schematically shows a side view of the checkout counter accordingto FIG. 7; and

FIG. 9 is a schematic workflow of a method of a classification deviceaccording to an embodiment.

DESCRIPTION OF EMBODIMENTS

FIG. 1 schematically shows a view from above of a checkout counteraccording to a first embodiment of the invention.

FIG. 1 shows an automated checkout counter 1 comprising a classificationdevice 2 for identifications of articles 3. The classification device 2comprises a weight sensor 4 for weighing the article 3, a memory unit 5comprising information of one or more articles, a processor 6 connectedto the memory unit 5 and to the weight sensor 4, and a firstspectroscopy sensor 7, from hereon denoted as a NIR or a VIS sensor 7depending on the associated wavelength interval, connected to theprocessor 6. The memory unit 5 comprises one or more first signaturescreated by the first spectroscopy sensor 7 or another spectroscopysensor (not shown), said first signature or which first signatures eachbeing connected to a corresponding article identity. The firstsignatures may be created immediately at the checkout counter by usingthe first spectroscopy sensor 7 or a second spectroscopy sensor (notshown) or by loading the memory with signatures created by aspectroscopy sensor which is not connected to the checkout counter 1.

In FIG. 1 the weight sensor 4 is shown placed before the first NIR orVIS sensor 7, which means that the first NIR or VIS sensor is arrangedto create a second signature after the article has been weighed, i.e.placed on the weight sensor and then weighed. The processor 6 is thenarranged to compare the second signature with the first signature toidentify the article 3 as one existing article identity in the memoryunit 5. The weight of the article will be used by the processor togetherwith the article identity to determine the price of the article.

As mentioned before a benefit of the invention is that the checkoutcounter automatically may identify all sorts of articles without anyneed for the customer to identify the article before the checkoutcounter, e.g. with a barcode. The NIR or VIS sensor is particularlyvaluable for identifying fruit and vegetables, and certain types of bulkarticles, since these articles before have required the customer toidentify the article and thereafter mark it due to the fact that sensorsusing cameras and image processing have not been able to determine thearticle identity.

Preferably, the weight sensor 4 comprises a conveyor scales 8 whichautomatically conveys the article and weighs it. The conveyor scale 8comprises a first conveyor belt 9 and a scales unit 10 on which theconveyor belt rests. The customer puts the article on the first conveyorbelt 9, wherein the scales unit 10 weighs the article and then the firstconveyor belt conveys 9 away the article 3. An alternative is that thefirst conveyor belt 9 conveys the article 3 to an appropriate position,stops and weighs, and then further conveys the article 3. At thecheckout counter 1, there are sensors arranged which give the processorinformation for control of the first conveyor belt 9 and the scales unit10.

The first NIR or VIS sensor 7, or a NIR or VIS sensor (not shown)connected to the processor 6, may be arranged to read an article andcreate the first signature during a learning procedure when the article3 already is identified for being able to be connected to the firstsignature.

FIG. 1 shows that the checkout counter 1 comprises an interactivedisplay unit 11 connected to the processor 6 for displaying at least onearticle identity. The display unit 11 is arranged for use by one user tobe able to approve the displayed information. If the first NIR or VISsensor 7 identifies the article 3, an image or a text is shown in thedisplay unit 11 and if the user finds the displayed information matchingthe article put in the checkout counter 1 the customer approves. Furtherinformation may be displayed, e.g. weight and price, wherein the userapproves the displayed if it is correct.

In addition to the weight sensor 4, the first NIR or VIS sensor 7 andthe display unit 11, FIG. 1 shows a second conveyor belt 12, and a thirdconveyor belt 13 for conveying the article 3. The direction of motion ofthe article on the conveyor belts is shown in FIGS. 1-8 by the referencesign x and an arrow which is shown in the direction of the motion. Thepurpose of several conveyor belts is that the article may be conveyed toa suitable final area where the article or articles may be picked up bythe user after payment. Another purpose is that the checkout counter 1may be designed in such a way that the weight sensor is placed after thefirst NIR or VIS sensor 7 (see FIGS. 3-8) or that the first NIR or VIS 7sensor may be placed in a way that the first NIR or VIS sensor 7 mayanalyze the article while it at the same time is being weighed. Thelatter is not shown since it should be obvious, with the background ofthe embodiments shown in FIGS. 1-8, how the first NIR or VIS sensor 7 isbeing placed relative the scales unit. Further purpose for havingseveral conveyor belts is if the checkout counter is provided withseveral sensors.

According to one embodiment of the invention the checkout counter 1 may,as a compliment to the first NIR or VIS sensor 7 and the weight sensor4, be provided with one or several further sensors which, if usedaccording to the invention, brings out the advantage of increasing thesecurity when identifying the article, but with a minimum use ofresources and time and energy. It should be mentioned here that anarticle 3 should preferably be identified in less than one second for anautomated checkout counter to be considered as user friendly by thecustomers.

The present invention also aims at solving the problem with optimal useof resources for several sensors by using the sensors according to somepredetermined combinations which provide high security when it comes toidentifying the product as well as high processing speed. Thecombinations also provides the advantage that when a given combinationis fulfilled, i.e. the sensor or the combined sensors provides apositive identification decision, the remaining sensors may bedisconnected or be controlled towards identification of a furtherarticle, which provides an optimum of the processing resources.

Even though the invention is mainly based on the weight sensor 4 and aNIR or VIS sensor 7 according to the above, FIG. 1 shows that thecheckout counter comprises several sensors which are connected in such away that a number of predetermined combinations comprising partial setof existing sensors, are enough for a positive decision, i.e.identifying the article 3. It should be mentioned that the embodimentwith further sensors provides a great amount of combinations and it istherefore not shown in separate figures because it would only lead to agreat amount of figures without increasing the understanding of theinvention.

The sensors may be switched on, i.e. activated, in sequences to be ableto find beneficial combinations or a partial set of or all the sensorsmay be active until one of the combinations provides a positivedecision, wherein one or more of the redundant sensors may bedisconnected. A positive decision is here when all sensors in thecombination have detected and indentified a predetermined property of anarticle, where the properties in combination together give the articlean identity. The identity may be determined through control against adatabase comprising properties of an amount of articles. The databasemay be stored in the memory unit according to what has previously beendescribed. Examples of properties are weight, size, color, shape,contour, marking with barcode and/or text and/or figure and/or pattern.

In order to provide a successful classification, an activated sensordetermines a measured signature of an article 3. The measured signatureis associated with the sensed signal, and may thus be a digitalrepresentation of a number of different article properties. Theprocessor is for this purpose configured to compare the measuredsignature with the digital reference signatures stored in the memoryunit 5, and to calculate a matching probability of a predeterminednumber of article identities.

The latter step is preferably performed by comparing the measuredsignature with all, or a subset of, the digital reference signatures ofthe memory unit and subsequently delivering the article identitieshaving the highest matching probabilities to a further classificationalgorithm, such as a BBN classifier.

The activated sensor is preferably the spectroscopic sensor 7, 24,either NIR, VIS, or both NIR and VIS, implemented as a single arrayspectrometer operating in a well defined wavelength interval accordingto what has been previously described. Hence, the measured signature isa digital representation of the reflectance spectrum in the givenwavelength interval.

The classification device may further use a further sensor beingselected from the group consisting of: a spectroscopy sensor 24, acontour sensor 14, a barcode reader 15, a symbol reading sensor 16, acolor texture sensor 17, a color histogram sensor 18, or a scale 4.

In case of using two or more different sensors thus providing two ormore different measured signatures, the processor 6 is configured todetermine specific article identities by comparing the matchingprobability from the different sensors 4, 7, 14, 15, 16, 17, 18, 24, andselecting the article identities having the highest matchingprobability. The article identities being identified by comparing themeasured signature of the first sensor, as well as the articleidentities being identified by comparing the measured signature of thesecond or further sensor, are thus transmitted to the BBN network forfurther analysis. In this situation the article identities may not beexactly the same for the different sensors and the comparing analysis.

The classification method may further include a step of comparing thehighest matching probability with an alarm threshold and, in case thehighest matching probability is below the alarm threshold, awaitingmanual input before proceeding.

In a preferred embodiment the method includes the step of comparing thehighest matching probability with two alarm threshold wherein, in casethe highest matching probability is below the lowest alarm threshold,the method awaits manual input from an attendant before proceeding, andin case the highest matching probability is above the lowest alarmthreshold but below the upper alarm threshold, the method awaits manualinput from a user before proceeding.

Further, if the scale 4 is used as a classifying sensor, the step ofcomparing the highest matching probability with an alarm thresholdcomprises the step of comparing the weight of the article with a weightinterval associated with the article identity corresponding to thereference signature having the highest matching probability.

In a further embodiment, if the contour sensor 14 is used as aclassifying sensor, the step of comparing the highest matchingprobability with an alarm threshold comprises the step of comparing theshape of the article with a shape interval associated with the articleidentity corresponding to the reference signature having the highestmatching probability.

In a yet further embodiment, is the barcode reader 15 is used as aclassifying sensor, the step of comparing the highest matchingprobability with an alarm threshold comprises the step of scanning abarcode of the article and comparing the information of the scannedbarcode with barcode information associated with the article identitycorresponding to the reference signature having the highest matchingprobability.

FIG. 9 shows a schematic flowchart of a classification algorithm 100which may be implemented by a classification device. The algorithmcomprises a number of steps for providing a successful classification ofarticles, e.g. in an automated checkout counter.

In step 102, a sensor determines a measured signature of an article. Themeasured signature is thereafter transmitted to a comparing unit, e.g.being incorporated in the processor 6, which comparing unit compares themeasured signature with the digital reference signatures in a step 104.Following this, in a step 106 a matching probability of a predeterminednumber of article identities is calculated.

As an optional step 110 a, the highest matching probability beingcalculated in step 106 is compared with an alarm threshold, and, in casethe highest matching probability is below the alarm threshold, themethod awaits manual input before proceeding.

As an alternative, the method 100 includes a step 110 b in which thehighest matching probability being calculated in step 106 is comparedwith two alarm thresholds wherein, in case the highest matchingprobability is below the lowest alarm threshold, the method awaitsmanual input from an attendant before proceeding, and in case thehighest matching probability is above the lowest alarm threshold butbelow the upper alarm threshold, the method awaits manual input from auser before proceeding.

Steps 110 a and 110 b may further comprise sub steps 112, 114, and 116respectively.

In step 112, the step of comparing the highest matching probability withan alarm threshold comprises the step of comparing the weight of thearticle with a weight interval associated with the article identitycorresponding to the reference signature having the highest matchingprobability.

In step 114, the step of comparing the highest matching probability withan alarm threshold comprises the step of comparing the shape of thearticle with a shape interval associated with the article identitycorresponding to the reference signature having the highest matchingprobability.

In step 116, the step of comparing the highest matching probability withan alarm threshold comprises the step of scanning a barcode of thearticle and comparing the information of the scanned barcode withbarcode information associated with the article identity correspondingto the reference signature having the highest matching probability.

According to an embodiment, the classification device comprises: aweight sensor 4, a first NIR or VIS sensor 7, a contour sensor 15 and/ora barcode sensor 15 and/or a symbol reading sensor 16 which uses opticalcharacter recognition and (machine) text interpretation and/or a colortexture sensor 17 and/or a color histogram sensor 18. The symbol readingsensor 16 is from hereon called OCR which is a general knownabbreviation of the English expression “Optical Character Recognition”.The sensors cooperate in such a way that if the sensors are activated inthe following combinations the remaining sensors are allowed todeactivate or not be activated at all, depending on the article beingidentified:

-   -   weight sensor 4 and contour sensor 14 and OCR 16, or    -   contour sensor 14 and OCR 16, or    -   weight sensor 4 and OCR 16, or    -   weight sensor 4 and color histogram sensor 18 and contour sensor        14, or    -   weight sensor 4 and contour sensor 14 and OCR 16, or    -   weight sensor 4 and color histogram sensor 18 and contour sensor        14 and color texture sensor 17 and OCR 16 and barcode sensor 15,        or    -   weight sensor 4 and contour sensor 14 and color texture sensor        17 and OCR 16 or    -   weight sensor 4 and barcode sensor 15, or    -   OCR 16.

The contour sensor 14 may comprise a camera for still or moving images,but may also comprise an object sensor. In FIG. 1, the contour sensor 14is shown as a linear camera which is placed in the slit between thefirst conveyor belt and the second conveyor belt which reads ahorizontal projected surface, in combination with an object sensor 20which consists of a vertical light curtain for reading the verticalprojection. The contour sensor 14 is connected to a unit for imageprocessing where the contour, i.e. a two dimensional projection of athree dimensional object, is checked against the properties in thedatabase.

In FIG. 1 an object sensor 20 comprising a light curtain devicevertically standing at the slit between the first conveyor belt and thesecond conveyor belt 12 is shown. The light curtain device comprises anumber of diodes with a transmitter on one side of the light curtaindevice and a receiver on the other side. Empirically it has been shownthat the preferred amount of diodes is on the order of 32 diodes andthat infrared diodes provide a good result. The invention is not limitedto 32 diodes which are based on infra-red light, but any other numberand frequency would work as long as the relative beams of the lightcurtain refract on different heights depending on the characteristics ofthe article and then provide information about the shape of the article.Since the article moves through the light curtain, a three dimensionalimage can be created by reading the light curtain at certain points oftime.

FIG. 1 shows that the barcode sensor 15 comprises a camera for still ormoving images, and that the symbol reading sensor 16 comprises a camerafor still or moving images, and that the color texture sensor 17comprises a camera for still or moving images, and a color histogramsensor 18 comprising a camera for still or moving images. The colorhistogram sensor 18 is preferably configured to detect three differentcolor properties, namely i) HUE values, blob values, and topologicalvalues. The invention is not limited to the use of one or more camerasas long as the corresponding sensors may provide information to theprocessor which then may provide information about the article identity.

The classification device 2 further includes an initial sensor 21 whichidentifies the article 3 by 100% and is arranged to be used duringlearning of the system by first identifying the article 3 andsubsequently conveying the article 3 through the classification device 2where all of the sensors identify properties of the article 3, whichproperties will subsequently be stored in a database for properties ofarticles.

In FIG. 1 the initial sensor 21 is shown as the barcode reader 15 whichis designed for manual use. The initial sensor 21 may however consist ofanother device which may provide the correct information to the memoryunit. A user may for example manually enter the product name of thearticle or other information, e.g. price and/or price per weight, foreach article. However, the barcode reader or another sensor admits thatthe system may be self-learned in a way that the articles are providedwith a barcodes or other identification and then be fed into the systemwhich automatically reads the identity and then lets the remainingsensors create its own signatures/recognition markers of the article.

FIG. 1 shows that the classification device comprises a handheld sensor22 which by 100% identifies the article and which may be used whenarticles are too big for the remaining classification device. Thehandheld sensor may be a barcode reader intended for manual use.

FIG. 2 schematically shows a side view of the checkout counter accordingto FIG. 1.

FIG. 3 schematically shows a view from above of the checkout counteraccording to other embodiments of the invention. FIG. 3 shows the samearrangement as in FIG. 1 but with a difference in order of weight sensor4 and the first NIR or VIS sensor 7. FIG. 3 shows that the weight sensor4 is placed after the first NIR or VIS sensor 7. With reference to FIGS.1 and 2, FIG. 3 shows that the different conveyor belts in the directionof movement of the conveyor belts are arranged after each other in thefollowing order; the second conveyor belt 12, the third conveyor belt,and the first conveyor belt 9 with the weight unit. In FIG. 3 thedisplay unit 11, as well as in FIGS. 1 and 2, is placed in connection tothe weight sensor 4 so that a user will be able to approve an article inconnection with the weighing. This is an advantage since the weight ofthe article is of importance for the price, which means that thecustomer would perceive a wrong price in connection to the weighing. Thewrong price could be due to an incorrect identification of the article,and the user may at this location of the weight sensor 4 have theopportunity to change to the correct article and thereby get the correctprice through a new or continued weighing connected to the correctarticle identity. In FIG. 3 the contour sensor 14 is placed between thesecond 12 and the third conveyor belt 13 and the first NIR or VIS sensor7 between the third conveyor belt 13 and the first conveyor belt 9.

FIG. 4 schematically shows a side view of the checkout counter accordingto FIG. 3.

FIG. 5 schematically shows a view from above of a checkout counteraccording to a third embodiment of the invention. FIG. 5 shows the samearrangement as in FIGS. 3 and 4, but with the addition of a camera 23placed in connection to the scales unit 10 to be able the capture animage of the article. The image should be displayed to the customer viathe display unit 11 so that the customer may be able to actively decideif the first NIR or VIS sensor has correctly identified the article. Theimage may also be displayed to a controller who is sitting at a distanceand who is able to decide if the first NIR or VIS sensor has read itcorrectly. In the case where several sensors are connected to thecheckout counter 1, the same reasoning applies regarding the image beingused by the customer or the controller to determine if the article hasbeen correctly identified. To speed up the identification process andalso make it more robust, the classification device may comprise afunction for uncertainty of the article identity, where many options aredisplayed to the customer via the display unit. The customer may thenchoose the correct option. In this context the above mentioned image maybe used together with the displayed information about the differentoptions to facilitate the identification, since the stored images of thearticle may be easier to compare with the image of the article then withthe article placed in the checkout counter.

FIG. 6 schematically shows a side view of the checkout counter accordingto FIG. 5.

FIG. 7 schematically shows a view from above of a checkout counteraccording to a fourth embodiment of the invention. FIG. 7 shows the samedevices as FIGS. 5 and 6, but with the addition of a furtherspectroscopy sensor 24 being arranged between the first conveyor belt 9and the second conveyor belt 13 to identify the product by means ofspectroscopy. Preferably, the further spectroscopy sensor 24 is acomplement to the first spectroscopy sensor 7, such that if the firstspectroscopy sensor 7 is a NIR sensor, i.e. detecting light having awavelength between 780 nm and 2500 nm, the further spectroscopy sensor24 is a VIS sensor, i.e. detecting light having a wavelength between 200nm and 1100 nm. Consequently, if the first spectroscopy sensor 7 is aVIS sensor the further spectroscopy sensor 24 is a NIR sensor. Thefurther spectroscopy sensor 24, in this case a VIS sensor, as well asthe first spectroscopy sensor 7, in this case a NIR sensor, may comprisea fiber cable which distributes light from the article to the respectivesensor. The fiber cable is a light guide acting as a probe. The VISsensor and the NIR sensor may both be connected to a separate fibercable which are arranged to converge into a common fiber cable whichdistributes light from the article to the VIS sensor and to the NIRsensor, respectively.

The probe, being provided to transmit light from the article 3 to thespectroscopy sensor 7, 24 is arranged to be moveable laterally relativea conveyor belt (9, 12, 13) of the device. The probe is connected to thespectroscopy sensor 7, 24, preferably being a single array spectrometerconfigured to operate in a wavelength interval according to what hasbeen described above. That is, the spectroscopy sensor may either be aVIS sensor, a NIR sensor, or a combined NIR and VIS sensor.

The classification device may further comprise a detector configured todetect the lateral position of the article 3 on the conveyor belt 9, 12,13, and a controller connected to said detector and being configured tomove said at least one sensor probe 7 to a position corresponding tosaid detected lateral position of the article 3. This may be done bydetecting the article 3 and calculating the lateral positioncorresponding to the mean value of the volume of the article. Thedetector may thus be camera, or any other optical sensor being connectedto an image processing device for calculating the mean value of thevolume.

The detector may further be configured to detect several positions ofthe article 3 as the article 3 is moving along the conveyor belt 9, 12,13 such that said at least one sensor probe 7 is sequentially moved tothe positions corresponding to said detected lateral positions of thearticle 3. Hence, if an article 3 is not aligned with the lateral orlongitudinal direction of the conveyor belt 9, 12, 13, the probe will bemoved during the article movement for providing measured signatures atdifferent positions of the article.

The classification device further comprises a light source forilluminating said article 3, wherein the emitted light has a wavelengthdistribution covering at least the operating wavelengths of thespectroscopy sensors 7, 24. Preferably, the light source is alsomoveable laterally relative said conveyor belt 9, 12, 13 such that thearticle 3 is sufficiently illuminated when the spectroscopy sensors areactivated.

The detector may further be triggered by an object sensor 20 configuredto detect the presence of an article 3 on said conveyor belt 9, 12, 13.Hence, the light source and the probes of the spectroscopy sensor(s) 7,24 may be positioned in an idle position until the object sensor 20triggers the detector, whereby the light source and the probes are movedto the position corresponding to the mean value of the article's volume.

The probes as well as the light source may be arranged on a linear stagearranged in a transverse direction of said conveyor belt 9, 12, 13 forproviding the moving functionality.

As the probes are moveable, the spectroscopy sensors, i.e thespectrometers, may be fixedly arranged at a remote position for reducingvibrations and other noise which may affect the spectroscopic analysisnegatively. However, the spectrometers may also be mounted to the movingstage.

The VIS sensor 24 may be arranged at a checkout counter according to anyone of FIGS. 1-6 and may be arranged at the first conveyor belt 9, thesecond conveyor belt 12, or the third conveyor belt 13.

When the VIS sensor 24 is turned on it is configured to operate in theintervals from 200 nm to 400 nm and from 660 nm to 1100 nm if it is usedin combination with the color texture sensor 17 and/or the colorhistogram sensor 18, but is configured to operate in the interval from200 nm to 1100 nm when the color texture sensor 17 and the colorhistogram sensor are disconnected. The processor 6 is arranged tocontrol the intervals of the VIS sensor depending on whether the colortexture sensor and/or the color histogram sensor are turned on or off.The VIS sensor 24 may be used in combination with any of the mentionedcombinations with reference to FIGS. 1-6, or only in combination withthe weight sensor 4 and the first MR sensor 7.

The VIS sensor 24 is connected to the processor 6 and to the memory unit5. The memory unit comprises one or several third signatures created bythe VIS sensor 24 or by another VIS sensor (not shown). The thirdsignature or the first signatures are each connected to a correspondingarticle identity. The third signatures may be created directly at thecheckout counter by using the VIS sensor 24 or another VIS sensor (notshown), or by loading the memory with signatures being created by a VISsensor not being connected to the checkout counter 1.

FIG. 7 shows that the weight sensor 4 is arranged after the VIS sensor24, which means that the VIS sensor 24 is arranged to create a fourthsignature by means of analysis before the article has been weighed, i.e.being arranged on the weight sensor 4 and subsequently being weighed.The processor 6 is thereafter arranged to compare the fourth signaturewith the third signature in order to identify the article 3 as anexisting article identity in the memory unit 5. The weight of thearticle 3 is used by the processor, together with the article identity,for determining the price of the article 3.

FIG. 8 schematically shows a side view of the checkout counter in FIG.7.

It shall be noted that the examples shown in FIGS. 1-8 are not limitingfor the invention, but only example of placements of sensors andconveyor belts. The checkout counter according to the invention maycomprise one or several conveyor belts. In case of several conveyorbelts, they may be angled towards each other and/or arranged forsplitting a flow of articles into partial flows etc. Adding more sensorsto the checkout counter than the first spectroscopy sensor and theweight sensor may be seen as further possibilities for improvedidentification and is thus a complement to the described embodiment withthe first spectroscopy sensor and the weight sensor. The additionalsensors may be placed in a vast amount of ways besides what is shown inFIGS. 1 to 8 for providing acceptable results within the scope of theinvention.

1.-16. (canceled)
 17. A classification device for identification ofarticles in an automated checkout counter, comprising: a memory unitcapable of storing digital reference signatures, each of which digitalreference signatures corresponds to an article identity, a processorconnected to the memory unit, and at least one sensor configured todetermine a measured signature of an article, wherein said processor isconfigured to compare said measured signature with the digital referencesignatures, and to calculate a matching probability of a predeterminednumber of article identities.
 18. The classification device according toclaim 17, wherein the at least one sensor is a spectroscopy sensor. 19.The classification device according to claim 17, wherein saidspectroscopy sensor is a single array spectrometer.
 20. Theclassification device according to claim 17, wherein said spectroscopysensor is configured to measure light having a wavelength between 200 nmand 2500 nm.
 21. The classification device according to claim 20,wherein said spectroscopy sensor is an infrared spectroscopy sensorconfigured to measure light having a wavelength between 850 nm and 2500nm.
 22. The classification device according to claim 20, wherein saidspectroscopy sensor is a visual spectroscopy sensor configured tomeasure light having a wavelength between 200 nm and 400 nm, and between660 nm and 1100 nm.
 23. The classification device according to claim 22,wherein said visual spectroscopy sensor is configured to measure lighthaving a wavelength in the complete interval between 200 nm and 1100 nm.24. The classification device according to claim 18, further comprisinga further sensor being selected from the group consisting of: aspectroscopy sensor, a contour sensor, a barcode reader, a symbolreading sensor, a color texture sensor, a color histogram sensor, or ascale.
 25. The classification device according to claim 17, comprisingat least two different sensors, wherein said processor is configured todetermine specific article identities by comparing the matchingprobability from the different sensors, and selecting the articleidentities having the highest matching probability.
 26. An automatedcheckout counter, comprising a classification device according to claim17.
 27. A method for classifying articles in an automated checkoutcounter, comprising the steps of: providing a classification devicecomprising a memory unit capable of storing digital referencesignatures, each of which digital reference signatures corresponds to anarticle identity, a processor connected to the memory unit, and at leastone sensor configured to determine a measured signature of an article,comparing said measured signature with the digital reference signatures,and calculating a matching probability of a predetermined number ofarticle identities.
 28. The method according to claim 27, furthercomprising: comparing the highest matching probability with an alarmthreshold and, in case the highest matching probability is below thealarm threshold, awaiting manual input before proceeding.
 29. The methodaccording to claim 28, further comprising: comparing the highestmatching probability with a two alarm threshold, in case the highestmatching probability is below the lowest alarm threshold, awaitingmanual input from an attendant before proceeding, and in case thehighest matching probability is above the lowest alarm threshold butbelow the upper alarm threshold, awaiting manual input from a userbefore proceeding.
 30. The method according to claim 28, wherein thestep of comparing the highest matching probability with an alarmthreshold comprises the step of comparing the weight of the article witha weight interval associated with the article identity corresponding tothe reference signature having the highest matching probability.
 31. Themethod according to claim 28, wherein the step of comparing the highestmatching probability with an alarm threshold comprises the step ofcomparing the shape of the article with a shape interval associated withthe article identity corresponding to the reference signature having thehighest matching probability.
 32. The method according to claim 28,wherein the step of comparing the highest matching probability with analarm threshold comprises the step of scanning a barcode of the articleand comparing the information of the scanned barcode with barcodeinformation associated with the article identity corresponding to thereference signature having the highest matching probability.